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ternating two-point experimental designs. The obtained results are useful in the cases 
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where it is known that variances of the subsequent observations are non-increasing, 
but the ratios of the decrease are not available to use sub-optimal or optimal Weighted 
Least Squares estimators. 
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1 Introduction 

If uncorrelated random variables have a common expected value and decreasing variances 
then, in agreement with common sense, the variance of a sample mean is decreasing with 
the number of observations. Surprisingly, this natural and desirable Variance Reduction 
Property (VRP) resulting from inclusion of less contaminated data is not shared in general 
by the Ordinary Least Squares (OLS) estimators of parameters. 

In fact, by generating random regression designs, it is easy to come across experimental 
designs where by adding additional observation with a smaller error variance, we in fact 
increase the variance of the augmented OLS estimators (we refer to Example [3] for a specific 
case). On the other hand, many such random designs indeed enjoy the VRP. In the case of 
homoscedastic models with identical variances, or when the covariance matrix is known up 
to a constant, the VRP is well known, with the Weighted Least Squares replacing the OLS 
in the latter case, and a fairly complete theory on optimal augmenting of the data is then 
available. We refer the reader to Sengupta (1995) and Bhaumik & Mathew (2001) for the 
most comprehensive recent results in this direction, and for further references to the related 
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literature. The case of unknown and decreasing variances appears difficult and seemingly has 
eluded the attention of researchers. This case however covers important applications in the 
theory of Asymptotic Statistics, where the asymptotic distributions of smoothed functionals 
are exactly of such a form. 

In the paper we derive (Theorem [2]) general conditions characterizing designs for which 
the OLS estimators of parameters retain the variance reduction property when new data 
with lower variances are included. These criteria can be easily used to iteratively construct 
sequences of experimental designs with the VRP. We found one interesting general case 
where the VRP remains valid: when n uncorrelated observations have equal variances of 
errors and the [n + 1)— th observation has a lower variance. Then the variance of the 
OLS estimator of parameters based on (n + 1) observations is lower than that based on n 
observations (Theorem [1]). Following the method of Theorem [2] in a particularly important 
linear model with two parameters: the intercept and slope, we show then that any design 
with the explanatory variable increasing with n has the VRP property (Theorem [3]), and 
that any two-point alternating experimental design has the VRP property (Theorem H]). All 
these results are non-trivial, novel and guarantee non-negativeness of the diagonal elements 
of matrices which themselves need not be positively definite. 

The paper is organized in the following way. In Section [2] we present the main results 
of the paper. Section [3] contains details of the straight line regression model and proofs of 
Theorems [3] and HI In the proofs we include only the main nontrivial steps and omit the 
detailed, often long, though elementary algebraic manipulations. Technical details can be 
found in Appendix \M In Section H] we include examples illustrating that the assumptions 
of Theorem [3] cannot be weakened. In Appendix |X] technical auxiliary results needed in 
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the proofs of the main theorems of the paper are presented. We also include Proposition 
m showing decomposition of variances in the general case of dependent random variables. 
Proposition |4] extends the decomposition flT4|) valid for uncorrelated random variables. We 
acknowledge using Scientific Workplace for deriving the results of this paper. 



2 Variance reduction in heteroscedastic models 



Consider the linear model 



Yo = Ao/3 + eo, 



(1) 



where Aq is an (n, k) design matrix, /3 is a {k, 1) column vector of regression parameters, 
and Yo = (Fi, Y2, . . . , F„) and eo G i?" are random vectors. We also consider an augmented 
model 



Yi = Ai/3 + ei, 



(2) 



where Ai 



An 



is an (n + 1, k) design matrix with a being a design of the (n + l)-th 



experiment, Yi = [Y^, F„+i] and e = [e^, e„+i] G -R""*"^ are random vectors such that 



E(ei)=0, Covie^) = ^u. (3) 

Apart from Example [2] and Proposition H] we will consider uncorrelated random variables 
with a diagonal covariance matrix Sn. Hence we have 



'11 



Soo 



o 



(4) 



a. 



n+l 
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where O is an {n, 1) vector of zeros and Sqo is a diagonal covariance matrix of Yq. To 
simplify notation we define 

An = AJAi and (5) 

Aoo = A'^Ao, (6) 

respectively. Both /3 and the covariance matrix Sn are assumed to be unknown and the 
design matrix Ai is assumed to be of full rank. The OLS estimators of parameters /3 based 
on models ([1]) and ([2]) are of the form 

A = Ar/AfYi (7) 

and 

(3o = Ao-qM^Yo, (8) 

and have covariance matrices Vn and Vqo given by 

Vn = Ar/Af SiiAi^r/ (9) 

and 

Ho = Aq-o^^SooAo^oo', (10) 

respectively. In the paper we derive conditions implying inequalities 

V^ii(^,0<K)o(^,0 for^G{l,2,...,fc}, (11) 

where Vu and Vqo are given by (|9]) and ( !T0|) . respectively. We will refer to property ( ITTj) as 
the variance reduction by augmenting the data, or, for short, as the VRP property. 
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It will be convenient to use the following notation throughout the paper 

dn+i = aAQQO^, and (12) 

qn+i = 1 + dn+i. (13) 

In the general case of a heteroscedastic model we have the following decomposition of 
variance Vu. 

Proposition 1 // the covariance matrix Sn and design matrix Ai in the model given by 
(j2]) are both of full rank, then for uncorrelated random variables Yi, i = 1,. . . ,n + l, we have 

Vii = Voo - al^iW - W^n, (14) 

where 

W^ = ^AooVaAoo\ (15) 

Qn+l 

H^n = ^oo'^ii^oo' - ^n ^ii^n , (16) 

Dii = A^DAq, and (17) 

D = Soo - Ci/. (18) 

In the particular case of iid random variables with common variance a^ the matrices D 
and Wii equal zero and then f lT^ reduces to the well known case of the Plackett (1950) 
updating formula for covariances of the LS estimators 

A^l = A^,' - W. (19) 

Let us note that in ( !T^ the matrix a'^^^W is non- negative definite and hence it always 
reduces the variances on the diagonal of Vu. However, in general, the matrix Wu need not be 
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positive definite, cf. Example[T]in SectionlU and for general designs it may even liave negative 
elements on the diagonal. Consequently, in the heteroscedastic case the variance of estimators 
of P can increase with an increase of the number of observations n, even when k and the 
parameters (3 are kept the same. As might be expected, if an+i > max {at, i = 1, . . . , n}, 
then the variance of (3i could be greater than that of Po- Therefore in the following we shall 
concentrate on the cases where the covariance matrices 

i:n = Diag{alal---al^,} (20) 

have non- increasing standard deviations ai,i = 1,2, . . . ,n + 1. Clearly, in these cases the 
matrix D given by flTSjl is also diagonal with non-increasing diagonal elements Di = af — 
^n+i — 0- ^s has been mentioned previously in the Introduction, decreasing o"j's alone do 
not guarantee the VRP, however. 

Nonetheless, we have the following surprising general case where the VRP is valid: if the 
first n observations have uncorrelated errors with the same variances and the error in the 
(n + l)-th uncorrelated experiment is lower than in the preceding cases, then the ordinary 
least squares estimators based on the augmented data have lower variances than in the case 
of n observations and the homoscedastic model. 

Theorem 1 If D = a'^I where I is the identity matrix then 

2 I n "n+1 



Wn=a'{2- ^ ;y ] W (21) 

i + a„_|_i ^ 



and, consequently, Wu is non-negative definite. 

Clearly Theorem [1] and Proposition [1] imply the following corollary. 
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Corollary 1 If random variables ¥±,¥2, ... ,¥„,+! are uncorrelated, var{¥i) = cxf for i = 
1,2, ... ,n and far(y„_|_i) = a^^i with af > a^^i then the OLS estimators have the variance 
reduction property. 

The proof of Theorem [1] can be obtained using fl57|) and fl58|) and some straightforward 
algebra, and is omitted. 

In the general uncorrelated case we have the following characterization of the non- negative 
diagonal elements of Wu, implying the VRP. 

Proposition 2 The diagonal element Wu{i,i) of Wn is non-negative for every diagonal 
covariance matrix (12U]) with ai non-increasing if and only if 

E((E^oo'(^,0^(j,OJ -\Y.A',^{i,l)A,U,l)\ J>0 for m = 1,2,..., n. (22) 

The following theorem follows immediately from Propositions [1] and [2j 

Theorem 2 The OLS estimator of the i — th component of f3 has the VRP if condition 
is met. 



Conditions (|22|) are in general not easy to verify. However, given the design matrix Ai 
of the first n experiments one can use flT9l) to find numerically designs a for the {n -\- 1)- 
st experiment satisfying inequalities fl22]) . Hence Proposition [2] provides a method for a 
sequential numerical construction of experimental designs having the VRP. Remark [3] in 
Section [3] provides more details in the case of a straight line linear model. 
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The linear model for a straight line regression has the following design matrix for n > 1. 



Ai 



1 hn+l 



1 hi 



1 K 



1 h 



(23) 



n+l 

The following two theorems show that in this important case the VRP remains a valid 
feature of two important classes of regression designs: with non-negative and increasing 
explanatory variables hi (Theorem |3]) and with two alternating values of the explanatory 
variable (Theorem H]). 

Theorem 3 // random variables Yi,i = 1,2, ... ,n + 1 are uncorrelated with non-increasing 
variances af and with increasing values of the explanatory variable hi > then the OLS 
estimators with design matrix ( 123|) have the variance reduction property ( TTTl) . 

Remark 1 In Theorem\^we consider models with a simple explanatory variable h. Clearly, 
any increasing function of h can be used here, as such a model, by applying a suitable change 
of parametrization, can be easily transformed to the case with a design matrix ( 123|) . 

Examples [T]-|3] of SectionlHshow that the assumptions of Theorem [3] cannot be weakened. 
This however does not contradict the fact that the VRP may also hold for many regression 
designs which do not meet the monotonicity assumption of Theorem |3l For example, it is 
well known that in the homoscedastic case two-point simple regression designs result in the 
lowest variance of the LS estimators of the slope and intercept. The following theorem shows 
that these two-point regression designs also have the variance reduction property. 
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Theorem 4 // random variables Yi,i = 1,2, . . . ,n + 1 are uncorrelated with variances af 
non-increasing with i, with two distinct values of the explanatory variable given by 

{b if i is odd 

c if I IS even 

then the OLS estimators (IHl) with design matrix ( l23l) and non-increasing variances (l20ll have 
the variance reduction property flTTl) . 

3 Variance reduction for a straight line model 

Consider the case of the straight hne model (!23l) . By Theorem [2] to prove the variance 
reduction property we need to verify that conditions (l22i) hold true. In this particular case 
fl22|) is equivalent to inequalities fH3|) and fH3|) of Condition C3. In Theorem [5] we show 
implication relations between Conditions CI — C7 given in Subsection I3.2[ In the proofs of 
Theorems [3] and H] we show that these conditions are met under the conditions specified in 
the corresponding theorems. We need however first to work out the structure of the diagonal 
elements of the matrix Wn. 

3.1 The Diagonal of Matrix Wu 

It will be convenient to use the following notation 

n n 

Si,n = ^hi, S2,n = ^h^, and (24) 

v;=^-f^y=-x:^.-^y. (25) 

n \ n J n ^ — ^ V n J 
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Notice that if Si^n 7^ then (J25ll is equivalent to 

S2,n Si^n _ Vn 



(26) 



Moreover, let us note that dn+i given by flT2|) reduces now to 



*n+l 



1 / {Si,n - nhn+iY 



n 



n^K 



+ 1 



(27) 



Matrices Aqq^ and W reduce in the present case to the following form 



4-1 - 



M^ = 7iy 



'J2,n —Sl^n 



(28) 



where 



1 / 1 



7w 



g„+i \n'^Vn 



(29) 



(30) 



and g„+i is given by flT3l) . We also get 



An' = ^-q' - W 



qn+i n?Vn 



and 



A^DAo = Dn 



<5ii <5 



Ih 



6 



Ih 'Jhh 



'J2,n+1 —Sl^n+1 
-Si„ + i n+1 






(31) 



(32) 



where Du is given by (I17p and Di are the diagonal elements of matrix D given by (ITS]) . By 
applying flT9|) we infer from f l28|l . f l29|l and fl3T|) that g„+i given by flT3l) equals 

(n + 1)2 K+i 



9n+l 



n^ 



K, ' 



(33) 
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where Vn+i stands for the variance of the design points given by ( l25l) with substitution 
n ^ n + 1. Clearly, since -Dj > the matrix AqDAq is non-negative definite. 

Remark 2 It may be interesting to note that (!33l) combined with (iT3ll and (1271) zs equivalent 
to the Welford (1962) updating formula for S2^n- 



It will be convenient to use the following notation. 



/ \ 

ai 


/ 


a2 




as 




04 




as 




[a,) 


\ 



C[n+lS2,n 
S2,n+1 
(ln+l^l,n 
Sl,n+1 

nqn+1 
n + 1 



and 






v 



al- 


\ 


aittg 


— 0204 


ttg — 


«I 


asos 


— 0406 


«^- 


«6 ^ 



(34) 



We hasten to note that in the present case conditions ( !22l) are equivalent to ( 145|) -( H6|) . 
respectively. However, to get conditions equivalent to ( 12^ in a simpler algebraic form we 
begin with the following diagonal form of Wu. 



Proposition 3 The diagonal elements of Wu are given by 



Wuihl] 



ai5u - '2,a2Sih + asS 



hh 



{n + lyv^+i 



(35) 



and 



Vrii(2,2) 



as^ii - 2a^5ih + a^5, 



hh 



(n + l)^K?+i 



(36) 



respectively, where 6ij are given by fl32l) . 
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Prior to further considering conditions guaranteeing non-negativeness of the diagonals of 
Wii we shall need some properties of polynomials driving the behavior of numerators in fl35p 
and dMD- 

3.2 Driving polynomials 

Let us introduce two closely related pairs of quadratic polynomials: {pi{h) , p2{h)) and 
{pi{h) , p2{h)) , referred to in the following as driving polynomials. We need them to for- 
mulate conditions equivalent to the variance reduction property. Polynomials pi{h) and 
P2{h) are given by 

Pi{h) = ai — 2a2h + a^h'^ and (37) 

P2ih) = 0:3 — 20:4/1 + a^h'^, (38) 

respectively, where ai, 02, • • • , as are given by (!34|) . In Lemmas [2] and [3] we derive their roots 
^1,1, ''"1,2 and r2,i, r2,2 given by (I64l)-( l67|) . respectively and show features needed in the sequel. 
Polynomials pi{h) and P2{h) are given by 

piih) = {h - rxi) {h - rx2) and (39) 

P2{h) = [h - r2,i) [h - r2,2) (40) 

and are obtained from pi{h) and P2{h) by dividing them by a^ and a^, respectively. 

3.3 Necessary and sufficient conditions for the VRP. 

In the following we consider statements C1-C7 and the corresponding assumptions under 
which they are equivalent to the non-negativeness of the diagonal of Wu. 
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CI The diagonal elements of matrix Wu given by (j35l) -( !36l) are non- negative for any vector 
of non-increasing and non- negative Z)j's. 

C2 For all non- increasing and non-negative Dj's 

ai6ii - 2a26ih + a^Shh > 0, and (41) 

asdu - ^a^Sih + a^hh > 0. (42) 

C3 For all non-increasing and non-negative Dj's 

n 

Y^ Di [ai - 2a2hi + a^hf) > and (43) 

n 

Y^ Di (as - 2a^hi + a^hf) > 0. (44) 

i=l 

C4 For each m G { 1 , . . . , n} 

«! — 2a2 — — + as — — > 0, and (45) 

m m 

as — 2a4 — ■ h as — — > 0. (46) 

m m 

C5 For each m G {1, ...,n} we have 

pi (^\ + asK^ > 0, and (47) 

\ m J 

P2 (^) + «5Kn > 0, (48) 

where polynomials pi{h) and P2{h) are given by ( 1371) and ( l38l) . respectively. 

C6 For each m G {1, ...,n} we have 

Pi (^] +V„,>0, and (49) 

\ m / 

P2 (^] + K. > 0, (50) 
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where polynomials pi{h) and P2{h) are given by ( l39l) and (l40l) . respectively. 

C7 For each m G {1, ...,n} 

Pl f ^"j + Kn > 0. (51) 

Theorem 5 l.IfVn>0 then statements CI — C5 imply each other. 

2. IfVn,<y3 o.iT'd as ore positive then CI — C6 imply each other. 

3. If hi, ... ., hn+i are non-negative and increasing then CI — C7 imply each other. 

Proof of Theorem [H 

Assuming that Vn> Q, the equivalence of CI, C2 and C3 is evident. LemmaU\ applied 
with Ui = ai — 2a2hi + a3h'^ implies the equivalence o/C3 and C4. By applying (1251) we get 
equivalence o/C4 and C5. Assuming additionally that as > and a^ > we can divide (H7I) 
&?/ as and (jUj) 6y as, respectively, without changing the direction of the inequalities. This 
shows equivalence o/ C5 and C6. If hi, . . . ,hn+i are non-negative and increasing then by 
(!68|) - (!70|) ■ Lemma\2\and Corollaries\^and\^hoth polynomials have the same roots ri^i = r2,i. 
Moreover, the root ri_2 of pi is larger than the root r2,2 of p2 and r2,2 > ''"2,1 ■ This implies 
that p2{h) > pi{h) fork G [?"i,i,Ti,2]- Hence for h G [^1,1, Ti, 2] (1491) implies fl50p . S'mce 6c»t/i 
Pi{h) and p2{h) are non-negative for h ^ [ri^i,ri^2] we get the equivalence 0/ C6 and C7. 
r/iis concludes the proof of Theorem (} 

Remark 3 Notice that the definition of ai 's given in flMl) implies that the left hand sides of 
inequalities (H5|) - (146|) m statement C4 are polynomials of degree 4 of variable hn+i . Hence 
it is clear that, given hi, ... , hn, the set of hn+i 's leading to the variance reduction equals 



VRP for OLS 16 

the intersection of the positivity regions of 2m polynomials of degree 4- No transparent 
characterization of these sets seems available, yet for numerical values of hi, . . . ,hn it is 
always possible to determine, at least numerically, which values of hn+i need to be avoided to 
retain the VRP of the design. This opens a way to practical sequential methods of determining 
designs with the VRP for decreasing variances. 

3.4 Proofs o Theorems [3] and [4]. 

Proof of Theorem [S 

By Theorem [5] it is enough to show that statement C7 holds true. 

Clearly (ISTIl holds true for m such that -^ < ri^i. Let mi be the smallest m such that 
-^^^ > rii. Since hi are increasing -^^^ is increasing with m and hence we have 

— '— > ''"i 1 for '^ ^ [^^l;''^l- 

m 

Inequalities ( 175|) imply that the interval [mi,n] is non-empty, it contains at least the right- 
hand end point n. 
Let us note that 

] +Vrn= [ ri,i ri,i + y- ^ - ri,2 - ri,i^ 

From Lemmas H] and [7| we infer that 



^1,1 



holds true for m = n. We will show that for m G [T7ii,n] the left hand side of (13. 4p is 
decreasing with m or, equivalently, that the expression on the right hand side of (j72l) is 
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positive. Hence, we need to show that 

hm+1 < —^ • (52) 

Let us fix 771 > mi and define function g{r) by 

9{r) = — . (53) 

Si^rn - mr 



By Corollary [2] we have 



ri,i {m) = g{hm+i) < ri,i < — —. (54) 

lib 



Notice that hm+i = g (^1,1 (fn)). Since for r < -^^^ the function g{r) is increasing (154|) implies 
(132]) . Hence we get that 

\ m J 
for any m G [?7ii,n]. This concludes our proof of Theorem [31 <) 
Proof of Theorem IH 

The design points hi in the present case, in contrast with the case considered in Theorem 
0, need he neither non-negative nor increasing and a^ can he negative, eg for b = —1, c = 1. 
By partUl of Theorem\^ we can rely only on the equivalence of statements CI — C5. To 
show that this two-point regression design has the VRP we need to consider two cases: for n 
even and for n odd. 

The case of even n. Condition WJ\\ reduces in the present case to 

/ St \ ^{n + 1) c^ {b — c) if m is even 

Pi ( -^^ j + a^Vm = I 

^^^ I i!n±i(^ + l)c2(5_c)2 zf mis odd 

> for m G [l,n] . 
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Condition ( l48ll reduces in the present case to 

/c \ i (n + 1) (6 — c) if m is even 

P2 f — — j + OlbVm = I 

^ ^ ^ I i!2i±i (^ + 1) (6 - of if m IS odd 

and clearly these expressions in are positive for distinct b and c and m G [1,"^] 

The case of odd n. Conditions (1471) - (HHl) reduce in the present case to 

'Sim\ I 2^(Sf^^ (^ ~ ^)^ ^f m is even 

Pi ( -I- ) + "sKn = \ 

2 m (n-1) V / ^ 

> for m G [l,n] 
and similarly 



m 



m 



i^("+i) 



• c \ Trn , ,,:j (6 — c) if m is even 

P2 I — ) + ttsKn = "^ 



-n-, 



2 m (n-1)" 

> for m G [l,n] . 

5*0, both for n even and for n odd the condition C5 is met and hence by PartUlof Theorem 
{5\ we conclude that the considered two-point design has the VRP. (} 



VRP for OLS 



19 



4 Examples and counterexamples 



Example 1 Matrix Wu need not be positive definite, even for decreasing af and increasing 
hi's. Let 



Ai 



1 0.62 

1 1.24 

1 1.80 

1 1.96 



and Ell = Diag {1.56, 1.26, 0.78, 0.28} . 



Tlien we have 



W 



11 



0.54605 -0.55859 
-0.55859 0.56787 



and the eigenvalues ofWu are given by 

-0.00174, 1.11566, 

respectively. So, the matrix Wu is indefinite, yet the diagonal elements are positive. 

Example 2 For correlated Yi, . . . ,Yn and uncorrelated Yn+i the VRP need not hold even 
for decreasing af and increasing hi 's. Let 



Ai 



- 






1 


7 
10 




I 


8 
5 


, s = 


1 


17 
10 




1 


19 
10 





9 5 _9_ 

^ 4 10 



5 6 2 
3 5 5 

5 1 1 

4-^4 



5 ^11 



S^S 







,/ 1 



where 0' = [0,0,0]. Then the diagonal of Sn equals {8.340 278,4.0025, 1.0325,0.2} and 



Vn — Vn+1 



-4.505 055 3.277313 



3.277313 -1.987787 
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ie. adding the fourth observation results in an increase of variance for both intercept and 
slope estimators. We refer to Proposition [H for a general decomposition in the case of 
correlated random variables. 

Example 3 For uncorrelated Yi, . . . , Yn, Yn+i the VRP need not hold even for decreasing 
af and increasing hi's i = 1, . . . ,n but with hn+i < hn. Let 

2 



Then we have 



A, 



1 


0.7 


1 


1.6 


1 


1.62 


1 


1.45 



and Sii 



10 



t 

5 



i 



yn ~ Vn+l 



-0.002 878 0.064 592 



0.064 592 -0.013 034 



A Auxiliary results. 



In derivation of our results we use the inversion formulae for block partitioned square ma- 
trices, included below for the convenience of reader: 



assuming that the inverse matrices exist (cf. Rao (1973), Problem 1.2.8 ), 

-I 
C B 



B^ D 



C-i + C-^B (D - B^C-'B)-^ B^C-^ -C'^B (D - B^C-^B) 



Tn-lu\-^ uTn-1 



[D-B^C-^ByB^C 



Tr^-lu\-^ 



[D - B^C-^B) 



(55) 



in the case of a nonsingular matrix C and two column-vectors B and D (cf. Rao 
(1973), Problem 1.2.7) 



.T^-l 



{C + BD^) ' = C-^ -- 



D^CB 



{C-'B) {D^C-') . 



(56) 
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Proof of Proposition [H Let us note that by applying ( |56l) and partitions (j4]) one 
gets from (Q the well known Plackett (1950) updating formula (1191) for covariances of the 
LS estimators in the iid case, which we use in the sequel. Let us also note that 

WA^AoW = j^^4^W, Wa^aW = ^"^'{"X ^^ (^7) 

1 + a^oo ^ -'- + '^^00 ^ 

Wa^aA^Q = aAg^a'^W and A^^a'^aW = aA^^a^W. (58) 

By applying next the block-matrix inversion formula (I55p and using (TTSj) one can derive (1141) 
with some algebra, omitted for the sake of compactness. 

The following simple lemma is pivotal for the present paper. 
Lemma 1 Let Ui,i = 1,2, ... ,n be given. Inequalities 

n 

hold true for all non-negative and non-increasing Di 's if and only if 

m 

^Ui>0 form = l,...,n. (60) 

Proof of Lemma [T], 

Clearly, by choosing Z^j = 1 for z = 1, . . . j-m and Di = ioi i > m we find that fl59l) 
implies fl60|) . To show that fl60|) implies fl59l) let us note that 

n 71—1 j n 

J2 D^Ui = J2 (Dj - ^,+i) J2^s + Dr,J2Us (61) 

i=l j=l s=l s=l 

and hence that the right hand side of (|6T|) is non-negative if (l60l) holds true. <) 

Remark 4 Property (160!) should not be confused with the majorization of Hardy, Littlewood 
& Polya (1952) because no ordering of Ui 's is here assumed. 
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Proof of Proposition [2l By ( fT6l) we have 

n / k \2 n / k X^ 

j=i \«=i / i=i \/=i / 

n / / k \'^n/k \2\ 

= E (E^o-o'(^'0^o(^'0) -E(E^n(^'0^o(^'0) pl^O 

Hence, in order to have 

Wii>0 

for every non- increasing D'j with D| = af — o"^_|_i, it is sufficient and necessary, by Lemma 
m to have (12^ . This completes the proof 



Lemma 2 VFe /iaw e 



Ai = a2 — aiOs 

= g^+i (S'2,„ - ^i^n/i^+i)^ hl^-^ > (62) 



anc? 



A2 = ^4 - 03^5 



d+i ('S'i,n - nhn+if > 0. (63) 



Proof. The proof of Lemma [2] follows immediately from flMl) and from the identity 

(a/3 - 5^f - {a" - 5^) (/^^ - 7') = ("7 - P^f ■ 

Hence, Lemma [2] implies that both polynomials pi{h) and P2{h) have roots. We will always 
assume that denominators differ from zero. 
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Lemma 3 Polynomials ( l37l) and ( |38l) have roots 



n,i = = = ^ ^ , (64) 

^3 0^3 0,3 — O'i (ln+lOl,n — Ol,n+l 

^1,2 = — + = \ = ^ — — ^ , (65) 

«3 ^3 C^S + ^4 Q'ji+l>-'l,n + -Jl.n+l 



and 



OLa, V A2 Cts — (24 "S*! „+i — gri+i5'i „ , . 

r2,i = = = — , 66 

"5 «5 05 - ag n + 1 - ng„+i 

04 V A2 as + 04 gn+iS*! „ + "S*! „+i , , 

^2 2 = 1 = = , (67) 

«5 «5 05 + ag n + 1 + ng„+i 

•where Ai anc? A2 are g'zwen 6y fl62|) and (!63|) . respectively. Moreover, we have 

ri,i = r2,i = ^ , (68) 

ri,2 - n,l = ^'^^n— 7 ^^ -2 , 

f;^ + [hn+i - 57^ j n^Vn {Si^n + 2nS'i,„ + nhn+i) + n^Si^n (hn+i - %^ j 

(69) 

and 

2^^^ (^ (^"+1 - %)' + (^' + 1) ^n) (/.„+! -^y + {2n + 1) K 

^l,n ((/in+l -^y + hn+1^ + {^U + 1) v}j ({h^^^ - ^j' + 2 (n + 1) K 

(70) 
Let us note that Lemmas [2H6] are valid for any hi, ... , h^+i for which the corresponding 
denominators differ from zero and they have been obtained by purely algebraic manipula- 
tions. For increasing non- negative sequences hi, ... , /i„+i Lemma [7] implies that the right 
hand sides of ( 168|) -( |69|) are positive. 
We shall need the following lemmas. 



ru2 - r2,2 
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Lemma 4 We have 

Pl ] +Vn = nVn 



"" ^ ^'^''" ^"+'''^ 5i,n (/in+1 - %) +(n/i„+i + 5i,„(2n + l))K 

(71) 
Lemma 5 For m <n we have 

- n,i + ^7 ^ - — r - ^1,1 + 



(m + 1) (%^ - ri,i) V "■^' -^i,™ - mri,i 
Lemma 6 For m < n we have 

'J2,m+l ~ \^l,m+l) hm+2 ^2,m '~ ^l,mhm+l 



■ (72) 



5*1,™+! — [m + 1) hm+2 Si^m — ""^^m+l 

m f %f - hm+i) +m{m+l)Vm 

= {hm+2 — hm+i) Yd r TTc 7 TlTT 7- (^3) 

\Ji,m — ii'm+i'm) \Ji,m+i — [rn + L)nm+2) 

We shall also need the following properties of the design points hi which can be easily 
derived using convexity arguments. 

Lemma 7 If n > 1 and hi are non-negative and increasing with i then ai, ... ,0:5 are posi- 
tive, 

hi < < -r- < hn, (74) 

n bi^n 

hi < ri 1 < < r2,2 < — -— < hn+1, (75) 

n n + 1 

and 

"F^ < n,2 < "F^ < hn+i- (76) 

1^1,12 *Jl,n+l 
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Proof of lemma [Tl 

Since 

2,n _ '^^ hi 

Sl,n ^ Sin 



we get 



Sl,n ^ S2n ^ r, T , 

< — — < max{hi} = hr, 



n Si^n 

Indeed, the left hand side inequality holds because heavier weights are assigned to hi 's of 
higher value. This implies (17^ . Moreover, we have 

Sl,nhn+1 ~ S2,n ST^ [hn+l ~ hi 



'^l,! 



nt 



E yif-n+1 ~ IH) , 

V" (h 7..^^ 



ihn+l — Si^n ~^ ^j=l (hn+l — hj) 

and hence 

hi < ri,i < 

n 

and the right hand side inequality holds because heavier weights are assigned to hi 's of smaller 
values. Clearly, we have hi < rii. Let us note that 

Qn+lS2,n + '-'2,n+l Qn+lSln '-'2,n , '-'l.n+l '-'2,n+l 

ri^2 = = — I • 

Qn+lSl,n + *-'l,n+l Qn+lSl,n + 'Jl.n+l '-'l,n Qn+lSl,n + Si^n+1 Si^n+1 

Hence 

S2,n S2,n+1 

-w^ < n,2 < ^ . 

Finally, we have 

Qn+lSl^n + "S*! „+i nQn+i Sin , Ti + 1 5*1 „+i 
T22 = = 1 

n + l + UQn+i n+ 1 + UQn+i u n + 1 + nqn+i n + l 

and hence 

Sin '-'l.n+l 

n n + l 
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Lemmas\^ and^ imply the following. 
Corollary 2 Let 

'^l,! {^) = -^ 7 = 9{hra+l), (77) 

where g is given by (1531) . If hi are non-negative and increasing then form < n then we have 

< Ti,i {m) < ri 1 (m + 1) < ri i [n) = ri_i. 

Corollary 3 For hi, ... , hn+i non-negative and increasing the values of all expressions on 
the right hand sides at (!68|) — (173!) are strictly positive. 

Let us finally note that in the case of correlated random variables we have the following 
decomposition extending ( 1141) onto the case of general correlated random variables. We keep 
here the same notation as in Proposition^ 

Proposition 4 If the covariance matrix Sn and design matrix Ai in a model given by ([2]) 
and (jlj) are of full ranks then we have 

Vu = Voo - al^iW - Wii + 1^22, (78) 

where 

W22 = A^^ {a^al^^^Ao + yl^ai,„+ia) A^l, (79) 

and ai^n+i is the {n,l) vector of covariances ofYn+i and {Yi,Y2, ...,Yn). 
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Proposition j^] can be shown in a similar way like the decomposition ( fT4l) of Proposition 
[I]. Proposition [7] shows explicitly how the difference between Vn and Vqo is affected by co- 
variances. We also refer to Example\^ where consequences of decomposition f l78|) are shown 
and where the non-zero correlations affect the VRP. 
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